The Comparative Analysis of Two RT-qPCR Kits for Detecting SARS-CoV-2 Reveals a Higher Risk of False-Negative Diagnosis in Samples with High Quantification Cycles for Viral and Internal Genes

The early detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) using the real-time quantitative polymerase chain reaction (RT-qPCR) as a gold-standard molecular tool has allowed to test and trace the viral spread and the isolation of COVID-19-infected patients. The detection capacity of viral and internal genes is an essential parameter to consider and analyze during the assay. In this study, we analyze the performance of the two commercial RT-qPCR kits used in Chile, TaqMan™ 2019-nCoV Control Kit v1 (Thermo Fisher) and MaxCov19 (TAAG Genetics), for the COVID-19 diagnosis from nasopharyngeal swab samples (NPSs). Our results show a lower sensitivity of the TAAG kit compared to the Thermo Fisher kit, even in the detection of SARS-CoV-2 mutations associated with its variants. This study reinforces the relevance of evaluating the performance of RT-qPCR kits before being used massively since those with lower sensitivity can generate false negatives and produce outbreaks of local infections.


Introduction
e COVID-19 pandemic has profoundly collapsed health systems in the world. Since it was declared a pandemic by the World Health Organization (WHO) on March 11, 2020 [1], more than 6.2 million people have died to date [2]. Although statistics currently indicate a decrease in infections and deaths in some countries with an increase in the vaccinated population [3], the appearance of new variants or genetic changes in the virus can undermine the protection provided by vaccines, increase infections, and generate sprouts locally and globally.
us, it is crucial to keep up a continuous search and testing to detect new possible infections by SARS-CoV-2. One of the main ways to control its spread is the early and effective detection of positive cases. For this, the molecular technique of real-time polymerase chain reaction (RT-qPCR) has been widely used and recommended for the diagnosis of COVID-19 disease [4]. ere are a series of commercial RT-qPCR kits available to carry out this technique in a massive way on the market. ese allow the detection of different target genes of SARS-CoV-2, such as envelope (E), nucleocapsid (N), spike (S), open reading frame 1ab (ORF1ab), or RNA-dependent RNA polymerase (RdRp), depending on the type of manufacturer [5]. However, important differences in the detection efficiency could compromise the kit performance and diagnosis. For example, the Gerbion GmbH & Co. diagnostics kit showed a detection efficiency of 49%, while the kit from SolGent Co., Ltd. manufacturer showed a detection efficiency of 92% in SARS-CoV-2 detection [6]. On the other hand, a study in India showed differences of up to 21% between the RT-qPCR kits used for the COVID-19 diagnosis [7].
In this study, we evaluate the performance of the RT-qPCR TaqMan 2019-nCoV Assay Kit v1 ( ermo Fisher) and MaxCov19 kit (TAAG Genetics), widely used for the detection and diagnosis of SARS-CoV-2 in NPSs of patients from the population of Santiago, Chile. We compared the Cq values for the detection of internal reference and viral genes. We found that Cq values are directly related to the SARS-CoV-2 detection efficiency and diagnosis. e results indicate a lower sensitivity of the TAAG kit than the ermo Fisher kit in detecting SARS-CoV-2 and its impact on the analysis and detection of SARS-CoV-2 variants. Our study emphasizes that the lower SARS-CoV-2 detection of an RT-qPCR kit could compromise the diagnostic and lead to uncontrolled spread.

Patient Samples. Nasopharyngeal swab samples (NPSs)
were collected in the hospitals that belong to the Central Metropolitan Health Service (SSMC) of Santiago, Chile. e swab samples were preserved and transported in a CIT-OSWAB ® transport kit (Reference code. 2118-0015; Citotest Labware Manufacturing Co., Ltd, Jiangsu, China). All the samples were processed before the first 24 hours after the sampling collection.

RNA Extraction.
According to the manufacturer's instructions, total RNA extraction was done using the Pure RNA Extraction Kit (TAAG Genetics, Santiago, Chile. Reference Code: TAGGE01003). e extracted RNA was used immediately for SARS-CoV-2 detection by RT-qPCR.

RT-qPCR Amplification Using TAAG Kit.
e detection of SARS-CoV-2 was made following a one-step strategy using the RT-qPCR MaxCov19 (TAAG Genetics, Reference code. TAGK01052). is assay was carried out using the N1 and E gene probes (loading 5 μl of total RNA extracted from NPSs per reaction), with some modifications. In brief, 2 μl of total RNA extracted from NPSs and 3 μl of nuclease-free water were loaded in each reaction. Positive control probes for viral genes and RNase P (internal reference gene) were included and assessed individually in the PCR plate. All the RT-qPCRs were performed on the Agilent AriaMx Real-Time PCR System (Agilent Technologies, Reference code. G8830A). Data were extracted using the Agilent AriaMx software.

PCR Efficiency and Limit of Detection (LoD).
To establish PCR efficiency and LoD for internal reference and viral gene probes assessed for TAAG and ermo Fisher kits, we ran RT-qPCRs using 10-fold serial dilutions. To get the maximum representation of values in the curve, we used for the 10-fold serial dilutions a reference pool made from randomized ten total RNA extracted from NPSs with a Cq value close to 20. e RT-qPCRs were carried out in triplicate according to the specific conditions indicated by the manufacturer and described above. All the RT-qPCRs were performed on the Agilent AriaMx Real-Time PCR System (Agilent Technologies, Reference code. G8830A). We determined the slope by linear regression in GraphPad Prism and defined the required levels for PCR efficiency (E%) > 95% and R-squared (R2) > 0.95. e primer efficiency was calculated according to the formula Efficiency (E%) � (10(-1/Slope))-1) * 100 [8]. To determine the detection limit, we select ten samples with Cq values close to 30. us, a standard curve determined the minimum detection limit for the TAAG RT-qPCR kit for each probe's amplification. e experimental TAAG RT-qPCR kit LoD was established based on the last dilution of all the triplicates amplified. We also considered the R2 (intended as a goodness-of-fit measure for linear regression) and the probe efficiency (closer to 100%, intended 100% as the optimum probe efficiency value). e experimental LoD was compared with the maximum Cq value suggested by the manufacturer and detailed in the kit brochure.

Data Representation and Statistical
Analysis. Paired twosided student's t-test was used for the Cq and RFU analysis of the reference gene. A one-way paired ANOVA test was used for the viral gene to analyze the differences between RFU and Cq values. Statistical software GraphPad Prism 8 was used to analyze and graph the data. A p < 0.05 was considered statistically significant.

Results
We evaluated the amplification profile using 2 and 5 μl (the last volume recommended by the manufacturer) of total RNA extracted from 91 random NPSs. irty-four samples were previously diagnosed as SARS-CoV-2 positive by the ermo Fisher kit. Importantly, for the RNase P internal reference gene probe (Figure 1(a)), no significant differences were observed in the mean Cq value of the samples when 2 μl (35.28 ± 3.74) and 5 μl (35.14 ± 5.37) of total RNA were loaded. is implies that the 87.9% (80/91) and 79.1% (72/ 91) samples, respectively, showed amplification with a Cq value ≤ 38 (considered as a suitable sample for analysis, according to manufacturer's instructions) (Figure 1(d)). For the N1 viral gene probe (Figure 1(b)), no significant differences were observed between their mean Cq values when 2 μl (28.14 ± 6.66) and 5 μl (28.82 ± 7.29) of total RNA were loaded. However, 91.2% (31/34) and 85.3% (29/34) of samples evaluated showed amplification with a Cq value ≤ 37 respectively, according to the manufacturer's recommendation for a positive SARS-CoV-2 sample. For the E viral gene probe (Figure 1(c)) significant differences were observed between their mean Cq values when 2 μl (30.35 ± 6.35) and 5 μl (43.06 ± 5.39) of total RNA were loaded. is implied that 91.2% (31/34) and 17.6% (6/34) of the samples registered amplification with a Cq value ≤ 37 (set following the manufacturer's recommendation for a positive SARS-CoV-2 sample) ( Figure 1(d)). Based on these results and previous antecedents for other RT-qPCR kits evaluated (Santibáñez et al., 2021), we defined 2 μl as the chosen loading volume for the TAAG RT-qPCR kit. e limit of detection (LoD) and the efficiency percentage (E%) of the probes for the TAAG RT-qPCR kit were determined, including the N1 (LoD: Cq value � 29.87, equivalent to 1152.2 copies/μl; E% � 115.4%) (Figure 1(e)) and E viral gene probes (LoD: Cq value � 31.09, equivalent to 496.26 copies/μl; E% � 118.5%) (Figure 1(f )). e LoD for the RNase P internal reference gene probe of the TAAG kit was not determined due to the lack of amplification when the samples were diluted (data not shown). In this way, the manufacturer's recommended criteria for considering a sample as a suitable sample for analysis: the Cq value is ≤ 38 for the RNase P reference gene probe. e manufacturer recommends a Cq value ≤ 37 both for N1 and E viral genes to consider the sample with SARS-CoV-2 positive diagnostic. e same analysis was performed for the ermo Fisher RT-qPCR kit. us, the LoD for the ORF1ab gene was 5.97 copies/μl (Cq � 37.15), 6.34 copies/μl for the N gene (Cq � 36.63); 10.28 copies/μl for the S gene (Cq � 35.72), and 3.51 copies/μl for the RNase P reference gene probe (Cq � 37.09) ( Supplementary Figures 1 and 2). Table 1 summarizes the LoDs for each RT-qPCR kit evaluated.

Amplification of Internal Reference and Viral Gene Probes of TAAG RT-qPCR Kit.
Once the LoD for the TAAG kit was determined, 91 NPS random samples were analyzed and compared their amplification parameters (Cq and RFU values) between the ermo Fisher and TAAG RT-qPCR kits. e paired comparison of the Cq values of the RNase P internal reference gene probe (Figure 2(a)) showed a significant difference between both RT-qPCR kits. e samples analyzed by the ermo Fisher kit showed a mean Cq value of 22.30 ± 2.62, while the TAAG kit showed 37.71 ± 4.07. Importantly, all these samples showed amplification for the RNase P internal reference gene probe, making them suitable for diagnostic when analyzed by the ermo Fisher kit ( Figure 2(b)). However, when TAAG analyzed the same samples, only 74.7% (68/91) of them could be considered suitable samples for diagnosis because their Cq values were ≤38 ( the maximum recommended Cq value for sample diagnosis). e paired comparison of RFU values obtained for both kits also showed significant differences (Figure 2(c)), with mean values of 5426 ± 596.1 and 921.8 ± 467.2 for the ermo Fisher and the TAAG RT-qPCR kits, respectively.
Also, the comparison of the viral genes of both kits was assessed. e paired comparison of the mean Cq values of viral gene probes of both RT-qPCR kits (Figure 2(d)) showed a significant difference, except between the S gene probe (31.38 ± 8.65) with the N1 gene probe (34.45 ± 9.39) and the E gene probe (35.77 ± 9.34) from the TAAG kit. e paired comparison of the RFU values showed significant differences between any of the viral gene probes between both RT-qPCR kits ( Figure 2(e)). No differences in the mean RFU value were registered between ORF1ab (3838 ± 649.4) and S viral gene probes (3311 ± 1910) for the ermo Fisher kit. e same case was also reported between N1 (1263 ± 1025) and E viral gene probes (1722 ± 1706) for the TAAG kit ( Figure 2(e)).   Figure 1: Determination of total RNA extracted from nasopharyngeal swab samples (NPSs) loading volume and the limit of detection (LoD) of SARS-CoV-2 probes using TAAG RT-qPCR kit. e upper section of the figure represents the Cq value paired comparison for total RNA extracted from NPSs loading volume determination. e analysis was made from the same NPSs using the manufacturer's recommended volume of 5 μl and 2 μl of total RNA extracted. In the graphs, the comparison was made for (a) RNase P internal reference, (b) N1, and (c) E viral gene probes. Each spot is a different analyzed sample for each volume condition (2 μl; 5 μl). On (a-c), the mean ± standard deviation (mean ± SD) for all the samples evaluated is indicated for each total RNA volume condition (2 μl; 5 μl). e line linking the spots indicate the paired result obtained for the same sample assessed using the two different volume conditions. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, paired two-sided student' t-test was applied (n � 80 random NPSs, n � 31 N1 gene, and n � 31 E gene random positive SARS-CoV-2 NPSs, * * * * p < 0.05). e lower section of the figure represents (d) the number of amplified samples using TAAG probes loading 2 and 5 μl of total RNA extracted. Limit of detection (LoD) for (e) N1 gene probe and (f ) E gene probe for SARS-CoV-2 detection. e analysis included 10-fold serial dilutions from a reference pool made from randomized ten total RNA NPS-extracted samples with a Cq value close to 20 (previously obtained using the ermo Fisher kit). For the LoD determination, a linear regression was performed.   When the criteria of Cq values recommended by the manufacturer for viral genes (Cq ≤ 37) and the internal reference gene (Cq ≤ 38) are applied to ensure the diagnostic of samples, the situation becomes more complex for the TAAG kit. For N1 and E probes, only 50% (10/20) and 45% (9/20), respectively, of positive SARS-CoV-2 samples were reported when the RNase P internal reference gene showed amplification (Figure 2(g)). However, when the experimentally determined LoD was applied, only 25% (5/20) of the samples were reported as positive for SARS-CoV-2 for TAAG viral gene probes (Figure 2(g)). As a consequence, the analysis with the N1 viral gene probe (Figure 2(h)) showed a 35% (7/20) and 60% (12/20) false-negative diagnosis when the manufacturer's recommendation (Cq ≤ 37) and experimental LoD (Cq ≤ 29.87) were applied, respectively. Similar results were obtained for the E viral gene probe (Figure 2 Because of the worrying percentage of false negatives observed for the TAAG kit (Figure 2(h)), we evaluated the performance of the TAAG kit with ten previously diagnosed SARS-CoV-2 positive samples with Cq values close to 30 using the ermo Fisher kit (Figure 3). e amplification of the RNase P internal reference gene probe (Figure 3(a)) showed significant differences between the mean Cq value of both RT-qPCR kits (Cq ermo Fisher � 24.19 ± 1.08; Cq TAAG � 40.19 ± 5.79). e RNase P internal reference gene probe amplification showed all the samples suitable for diagnosis by the ermo Fisher kit (Figure 3(b)). However, when TAAG analyzed the same samples, only 60% (6/10) of these could be considered suitable samples for diagnosis because their Cq value was ≤ 38, the maximum Cq value recommended for diagnosis. When the amplification of the viral genes was analyzed, we registered differences between the mean Cq value of ORF1ab (31.16 ± 0.87), N1 (37.34 ± 5.74), and E (42.20 ± 6.13) viral gene probes (Figure 3(c)). e percentage of these SARS-CoV-2 positive samples diagnosed by the viral gene probes of the TAAG kit (Figure 3(d)), following the manufacturer's recommendations, was 60% (6/10) and 30% (3/10) for N1 and E viral gene probes, respectively. When the criteria of internal reference gene amplification (Cq ≤ 38) ((+) RNase P) were applied, both viral probes showed 30% (3/10) of samples as positive for SARS-CoV-2 (Figure 3(d)). is implies for the TAAG kit a 40% (4/10) and 70% (7/10) false-negative samples were diagnosed for N1 and E gene probes, respectively (Figure 3(e)). However, no one positive sample was detected when the experimentally determined LoD was applied to TAAG viral gene probes (Figure 3(d)). us, 100% (10/10) of false-negative diagnoses were recorded using the TAAG RT-qPCR kit on samples previously detected with a Cq value close to 30 by the ermo Fisher kit (Figure 3(e)).

Discussion
Currently, multiple RT-qPCR kits are commercially available for SARS-CoV-2 detection. However, they lack appropriate clinical analysis and validation for use in the COVID-19 pandemic. Various studies of comparative analysis and proofs of RT-qPCR kits have previously been reported. For example, comparisons of kits used for COVID-19 diagnosis in Korea showed differences of up to 25% in the sensitivity of detection of SARS-CoV-2 [9]. In China, some kits used have 83% and 94% sensitivity, according to Sansure Biotech Inc. and BioGerm Medical, respectively, both with 100% specificity without cross-reaction in the detection of other viruses [10]. On the other hand, Eberle et al. 2021, analyzed nine commercial RT-qPCR kits used for the diagnosis of COVID-19 in Bavaria, Germany, with differences of up to 50% in detection sensitivity [6]. Also, a study using RNA from cell cultures identified differences in the number of viral copies detected in eleven commercial RT-qPCR kits, which varied between 3.3 and 330 RNA copies, where one also jointly noticed another human coronavirus (MERS), revealing nonspecificity in the diagnosis for SARS-CoV-2 [11]. All these studies strongly support the importance of an in-depth analysis of the performance of the RT-qPCR kits used to control the current pandemic, which translates into better or worse control and traceability of infected patients. However, to date, no study has reported an analysis of the RT-qPCR kits widely used to diagnose SARS-CoV-2 in the Chilean population. Furthermore, none of the previous studies has considered a deep analysis of the RT-qPCR parameters of the cellular internal reference gene (only viral target genes are mainly considered). e amplification of the internal reference gene is essential but underestimated data in an RT-qPCR analysis to ensure the accuracy of negative and positive results in clinical diagnosis [12].
We found that the TAAG kit showed serious difficulties in amplifying the internal and viral reference genes, along with both high Cq and low RFU values. is was reflected in a difference of up to 40% in SARS-CoV-2 detection where the ermo kit showed a detection efficiency of 100% for N and ORF1ab genes. In addition, the samples analyzed with the ermo Fisher kit showed RFU values up to 6 times higher than those observed with the TAAG RT-qPCR kit,   Figure 3: Comparative analysis between ermo Fisher and TAAG RT-qPCR kits with a known Cq value close to 30 (for ORF1ab gene probe, ermo Fisher). e comparison was made for the same NPSs using the previously optimized volume of 2 μl of total RNA extracted. e upper section of the figure represents the analysis for the RNase P internal reference gene probe amplification from 10 NPSs previously diagnosed as SARS-CoV-2 positive by the ermo Fisher RT-qPCR kit with a Cq value close to 30. (a) Paired comparison of Cq value of RNase P internal reference gene probes from ermo Fisher and TAAG kits. In graph, the horizontal red-dotted line represents the LoD for RNase P reference gene probe (Cq � 36.9) by ermo Fisher kit, and the horizontal blue-dotted line represents the maximum recommended Cq value (Cq � 38) by the manufacturer for suitable sample diagnosis using TAAG RT-qPCR kit RNase P reference gene probe. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, a paired two-sided student's ttest was applied (n � 10 NPSs, * * * * p < 0.05). (b) Number of suitable samples diagnosed with a Cq value within the detection range of the internal reference gene probes. e lower section of the figure represents the analysis for the viral gene probe amplification from 10 NPSs previously diagnosed as SARS-CoV-2 positive by the ermo Fisher RT-qPCR kit with a Cq value close to 30. (c) Paired comparison of Cq value of viral gene probes from ermo Fisher (ORF1ab viral gene probe) and TAAG (N1 and E viral gene probes) kits. In graph, the horizontal colored-dotted lines blue, green, and red, respectively, represent the experimental LoD for N1 gene (Cq � 29.87), E gene (Cq � 31.09), and ORF1ab gene (Cq � 37.14) viral gene probes. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, paired one-way ANOVA-test with multiple comparison test analysis was applied. (n � 10 NPSs, * * * * p < 0.05). In graphs (a) and (c), the mean ± standard deviation (mean ± SD) is indicated for the viral gene probe amplification obtained for all the samples evaluated. e line linking the spots indicate the paired result obtained for the same sample assessed by both RT-qPCR kits. (d) Summary of the number of positive NPSs diagnosed using TAAG RT-qPCR kit considering the RNase P internal control amplification. In the graph, "(+) SARS-CoV-2" represents a positive virus diagnosis, and "(+) RNase P" represents the amplification of the internal reference gene. (e) Summary of the percentage of false negative NPSs diagnosed by TAAG RT-qPCR kits applying brochure or experimental LoD criteria.
indicating less inhibition in the RT-qPCR reaction. Based on these data, it is not surprising that the diagnosis of NPSs includes a high number of false-negative diagnoses by the TAAG kit. is is because the TAAG kit has an LoD of over 1000 viral copies/μl for N1 and about 500 viral copies/μl for the E gene, which is equivalent to a sensitivity of up to 100 times less compared to the ermo kit. Fisher. Consequently, the detection of SARS-CoV-2 by the TAAG kit was seriously compromised.
Our report was closely related to that previously described [13] for the ermo Fisher RT-qPCR kit, which has shown a sensitivity of >96% in detecting SARS-CoV-2. is evidence was also according to other reports showing the optimal performance of the ermo Fisher kit for the detection of low viral loads (Cq > 30) [13], and in the detection of SARS-CoV-2 in samples without a previous total RNA extraction process (which decreases the amount of free RNA in the sample to be analyzed) [14]. is background confirms the high sensitivity of this kit for the diagnosis of COVID-19. On the other hand, most of the comparative investigations of different RT-qPCR kits for the diagnosis of COVID-19 focus only on the amplification of the SARS-CoV-2 gene, omitting the information of the reference gene, which can invalidate a result if it presents faulty parameters [15][16][17]. For example other kits use, for example, exogenous internal controls (e.g., EAV; equine arteritis virus, etc.). However, this is only an intrinsic control and is not related to quality and a competent sample [18]. In relative expression analyses, the reference gene is essential to normalize the expression levels of target genes [12]. In diagnosis, it is a solid basis to validate the proper process of total RNA extraction and subsequent amplification of the SARS-CoV-2 genes of interest used (ORF1ab, N, S, E, and RdRp). Due to the nonamplification of an internal gene, the results may be invalidated [19]. e comparative analysis between the ermo Fisher and TAAG RT-qPCR kits revealed differences in the RFU values obtained for the same set of samples, indicating differences in kit performance and specifically in the degree of inhibition for each reaction [20]. A low RFU value could even be mistaken for a background signal [21]. In this way, the incorporation of RFU values analysis supports our results. For this reason, we suggest that it is also a parameter that should be carefully considered to decide if a sample is suitable for issuing a diagnostic result. erefore, these findings indicate limitations on the use of the TAAG kit for the diagnosis of COVID-19. As a consequence, the misidentification of positive cases also compromises the identification of circulating SARS-CoV-2 variants in the population (like the gamma variant). RT-qPCR kits were developed to detect characteristic SNV associated with SARS-CoV-2 variants. However, samples must be diagnosed as positive using an RT-qPCR kit before this analysis. erefore, it is essential to avoid using low-sensitivity RT-qPCR kits since this could include false-negative diagnoses and ambiguities in the clinical diagnosis. is is a matter of critical concern, especially at this epidemiological stage of the pandemic with the presence of more contagious variants such as omicron [22].
Our results do not consider the specificity of the analyzed kits. In this regard, some reports indicate cross-reaction with other types of viruses [11], which is an aspect to be considered by the manufacturers of commercial RT-qPCR kits.
us, we suggest an exhaustive analysis of all commercial kits before they are used for the best control of this and other infectious diseases

Conclusions
Our results indicate that (1) the TAAG kit shows low Cq values for internal and viral genes of SARS-CoV-2 due to low detection of viral copies/μl . (2) e TAAG RT-qPCR kit negatively impacts the detection for the gamma variant of SARS-CoV-2 compared to the ermo Fisher kit. (3) e massive use of kits with a high risk of diagnosis of false negatives could jeopardize the application of public policy measures to control the pandemic.

Data Availability
e data used to support the findings of this study may be released upon direct reasonable request to the corresponding author, who can be contacted at e-mail felipe.reyes.l@usach.cl and felipe.reyes@uab.cat.

Consent
e study contemplated the use of an oral informed consent delivered to the patients by the health centers responsible for the collection of the nasopharyngeal swab samples used for SARS-CoV-2 testing. Disclosure e funders had no role in study design, data collection and analysis, the decision to publish, or the preparation of the manuscript.